Electronic device and controlling method of electronic device

ABSTRACT

Provided is an electronic device including a first sensor configured to sense a user; a second sensor configured to sense light incident from an external light source; a mirror display; a memory; and a processor configured to: based on the first sensor sensing the user in front of the mirror display, obtain information about a first area in which a first image corresponding to a reflection of the user is located on the mirror display; determine a second area for displaying content to be provided to the user based on the information about the first area; based on information about light being obtained through the second sensor, determine luminance of the second area based on the information about the light; and control the mirror display to display the content with the determined luminance on the second area.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a bypass continuation of International ApplicationNo. PCT/KR2022/003665, filed on Mar. 16, 2022, which is based on andclaims priority to Korean Patent Application No. 10-2021-0034481, filedon Mar. 17, 2021, in the Korean Intellectual Property Office, thedisclosure of which is incorporated by reference herein in its entirety.

BACKGROUND 1. Field

This disclosure relates to an electronic device and a controlling methodthereof, and more particularly, to an electronic device that may improvevisibility of reflected content viewed on a mirror display and acontrolling method thereof.

2. Description of Related Art

An electronic device equipped with a mirror display capable ofdisplaying content while performing a mirror function has beendeveloped. Specifically, the mirror display reflects light incident onthe mirror display from an external light source to perform a mirrorfunction, while displaying content/information stored in the electronicdevice.

However, when an image corresponding to a reflection of a user in frontof the mirror display is reflected off of the mirror display accordingto the mirror function for viewing by the user, there may be a problemin that the visibility of the content/information displayed by theelectronic device may be degraded by light incident from the externallight source and/or the user image.

Accordingly, there is need to improve the visibility of thecontent/information displayed on the mirror display in consideration ofthe effect of light incident from the external light source and/or theuser image.

SUMMARY

Provided are an electronic device that may improve visibility of contentdisplayed on a mirror display and a controlling method thereof.

According to an aspect of the disclosure, a first electronic device mayinclude a first sensor configured to sense a user; a second sensorconfigured to sense light incident from an external light source; amirror display; a memory; and a processor configured to: based on thefirst sensor sensing the user in front of the mirror display, obtaininformation about a first area in which a first image corresponding to areflection of the user is located on the mirror display, determine asecond area for displaying content to be provided to the user based onthe information about the first area, based on information about lightbeing obtained through the second sensor, determine luminance of thesecond area based on the information about light, and control the mirrordisplay to display the content with the determined luminance on thesecond area.

The information about the first area my include information about a sizeof the first area and information about a position of the first area.The processor may be further configured to determine a position and asize of the second area based on the information about the size of thefirst area and the information about the position of the first area.

The information about light may include information about a direction oflight and information about an intensity of light. The processor may befurther configured to: determine the position and the size of the secondarea based on the information about the first area and the informationabout the direction of the light, and determine luminance of the secondarea based on the information about the intensity of light.

The first sensor may include an image sensor, and the processor may befurther configured to: obtain a second image corresponding to the userthrough the image sensor, obtain information about a pose of the user byinputting the second image into a trained neural network model, anddetermine the position and the size of the first area based on theinformation about the pose of the user.

The processor may be further configured to identify types of a pluralityof objects within the content, respectively, by inputting data about thecontent to a trained neural network model, and determine luminance ofareas corresponding to each of the plurality of objects in the secondarea based on the information about light and the types of each of theplurality of objects.

The processor may be further configured to determine luminance of thesecond area based on the information about light, the information abouttypes of the content, and information about brightness of the content.

The processor may be further configured to: determine a position of auser interface to control the content based on the information about thefirst area and the information about the second area, and control themirror display to display the user interface at the determined position.

The processor may be further configured to: identify whether the user isa pre-registered user based on user data stored in the memory, based onthe user being the pre-registered user, identify the content based onthe user data corresponding to the pre-registered user, and control themirror display to display the identified content on the second area.

The electronic device may further include a communication interface. Theuser data may be health-related data received from an external devicethrough the communication interface, and the content may includerecommended content corresponding to the health-related data.

The processor may be further configured to: process the content so thata boundary area between the first area and the second area is blended,and control the mirror display to display the processed content on thesecond area.

According to another aspect of the disclosure, a method of controllingan electronic device comprising a mirror display may include based on auser on in front of the mirror display being sensed, obtaininginformation about a first area in which a first image corresponding to areflection of the user is located on the mirror display; determining asecond area for displaying content to be provided to the user based onthe information about the first area; based on information about lightbeing obtained through a second sensor, determining luminance of thesecond area based on the information about light; and displaying thecontent with the determined luminance on the second area.

The information about the first area may include information about asize of the first area and information about a position of the firstarea. The determining the second area may include determining a positionand a size of the second area based on the information about the size ofthe first area and the information about the position of the first area.

The information about light may include information about a direction oflight and information about an intensity of light. The determining thesecond area may include determining the position and the size of thesecond area based on the information about the first area and theinformation about the direction of the light. The determining luminanceof the second area comprises determining luminance of the second areabased on the information about the intensity of light.

The method may further include obtaining a second image corresponding tothe user; obtaining information about a pose of the user by inputtingthe second image to a trained neural network model; and determining theposition and the size of the first area based on the information aboutthe pose of the user.

The content may include content including a plurality of objects, and thdetermining the second area further may include identifying types of theplurality of objects, respectively, by inputting data about the contentto a trained neural network model; and determining luminance of areascorresponding to each of the plurality of objects in the second areabased on the information about light and the types of each of theplurality of objects.

According to yet another aspect of the disclosure, an electronic devicemay include a first sensor configured to sense a user; a second sensorconfigured to sense light incident from an external light source; amirror display configured to provide a visible reflection of the userwhile also displaying content; a memory; and a processor configured to:based on the first sensor sensing the user in front of the mirrordisplay, obtain information about a location of a reflection of the userfrom a perspective of the user, determine a displaying area fordisplaying content to be provided to the user based on information aboutthe location of the reflection of the user, and control the mirrordisplay to display the content on the displaying area.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the present disclosure will be more apparent from thefollowing description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a block diagram of a configuration of an electronic device,according to an embodiment of the disclosure;

FIGS. 2, 3, and 4 shown an operation of an electronic device, accordingto an embodiment of the disclosure;

FIG. 5 is a diagram of an operation of the electronic device fordetermining a second area, according to an embodiment of the disclosure;

FIG. 6 is a diagram of an operation of the electronic device fordetermining a first area according to an embodiment of the disclosure;

FIG. 7 is a diagram of an operation of the electronic device fordetermining brightness differently for each object included in thecontent, according to an embodiment of the disclosure;

FIG. 8 is a diagram of an operation of the electronic device fordisplaying a user interface for controlling content, according to anembodiment of the disclosure;

FIG. 9 is a block diagram illustrating details of an electronic deviceaccording to an embodiment of the disclosure; and

FIG. 10 is a flowchart of a method of controlling an electronic device,according to an embodiment of the disclosure.

DETAILED DESCRIPTION

Hereinafter, embodiments of the disclosure will be described withreference to the accompanying drawings. However, it may be understoodthat the disclosure is not limited to the embodiments describedhereinafter, but also includes various modifications, equivalents,and/or alternatives of the embodiments of the disclosure. In relation toexplanation of the drawings, similar drawing reference numerals may beused for similar constituent elements.

When it is decided that a detailed description for the known art relatedto the disclosure may unnecessarily obscure the gist of the disclosure,the detailed description may be omitted.

In addition, the following embodiments may be modified in many differentforms, and the scope of the technical spirit of the disclosure is notlimited to the following examples. Rather, these embodiments areprovided so that this disclosure will be thorough and complete, and willfully convey the technical spirit to those skilled in the art.

The terms used herein are to describe certain embodiments and are notintended to limit the scope of claims. A singular expression includes aplural expression unless otherwise specified.

In this specification, the expressions “have,” “may have,” “include,”“may include,” or “comprise” or the like represent presence of acorresponding feature (for example: components such as numbers,functions, operations, or parts) and does not exclude the presence ofadditional feature.

In this document, expressions such as “at least one of A [and/or] B,” or“one or more of A [and/or] B,” include all possible combinations of thelisted items. For example, “at least one of A and B,” or “at least oneof A or B” includes any of (1) at least one A, (2) at least one B, or(3) at least one A and at least one B.

As used herein, the terms “first,” “second,” or the like may denotevarious components, regardless of order and/or importance, and may beused to distinguish one component from another, and does not limit thecomponents.

Description of a certain element (e.g., first element) being“operatively or communicatively coupled with/to” or being “connected to”another element (e.g., second element) should be understood to mean thatthe certain element may be connected to the other element directly orthrough still another element (e.g., third element).

On the other hand, description of a certain element (e.g., firstelement) being “directly coupled to” or “directly connected to” anotherelement (e.g., second element) may be understood to mean that there isno element (e.g., third element) between the certain element and theanother element.

Also, the expression “configured to” used in the disclosure may beinterchangeably used with other expressions such as “suitable for,”“having the capacity to,” “designed to,” “adapted to,” “made to,” and“capable of,” depending on cases. Meanwhile, the term “configured to”does not necessarily mean that a device is “specifically designed to” interms of hardware.

Instead, under some circumstances, the expression “a device configuredto” may mean that the device “is capable of” performing an operationtogether with another device or component. For example, the phrase “aprocessor configured to perform A, B, and C” may mean a dedicatedprocessor (e.g., an embedded processor) for performing the correspondingoperations, or a generic-purpose processor (e.g., a central processingunit (CPU) or an application processor) that can perform thecorresponding operations by executing one or more software programsstored in a memory device.

Terms such as “module,” “unit,” “part,” and so on are used to refer toan element that performs at least one function or operation, and suchelement may be implemented as hardware or software, or a combination ofhardware and software. Further, except for when each of a plurality of“modules,” “units,” “parts,” and the like needs to be provided in anindividual hardware, the components may be integrated in at least onemodule or chip and be provided in at least one processor.

Various elements and areas in the figures are shown out of scale.Accordingly, the scope of the disclosure is not limited by the relativesizes or spacing drawn from the accompanying drawings.

Hereinafter, with reference to the attached drawings, embodiments willbe described in detail so that those skilled in the art to which thedisclosure belongs to can easily make and use the embodiments.

FIG. 1 is a block diagram of a configuration of an electronic deviceaccording to an embodiment of the disclosure.

As shown in FIG. 1, an electronic device 100 according to an embodimentmay include a mirror display 110, a sensor 120, a memory 130, and aprocessor 140.

The electronic device 100 may be implemented as a mirror display 110having a shape such as a stand-type mirror. However, the electronicdevice 100 is not limited to the type and shape of the electronic device100 of the disclosed embodiments, and the electronic device 100 may beimplemented as a TV or a smartphone.

The mirror display 110 may be configured to display content whileperforming a mirror function. That is, an image corresponding to anobject disposed in front of the mirror display 110 may be off of themirror display 110, and content stored in the memory 130 may bedisplayed like a conventional display while an image corresponding to anobject disposed on the front surface of the mirror display 110 isdisplayed. The mirror display 110 may include a display panel and amirror panel disposed on the display panel, and may further include atouch screen panel disposed on the mirror panel.

When the light incident from the external light source is reflected onthe mirror display 110, an image corresponding to an object positionedin front of the mirror display 110 may be reflected off of one area ofthe mirror display 110. In other words, if the object is positioned infront of the mirror display 110, a reflection of the object may beprovided on one area on the mirror display 110. When a user input fordisplaying content is received or an event for displaying content isgenerated, the mirror display 110 may display content stored in thememory 130. The content stored in the memory 130 may include not onlycontent obtained by the electronic device 100 and stored in the memory130, but also content received from an external device and stored in theelectronic device 100.

The sensor 120 may be configured to sense various information within oroutside the electronic device 100. The sensor 120 may include one ormore sensors 120, and may include a first sensor 120 for sensing a userand a second sensor 120 for sensing light incident from an externallight source. The first sensor 120 may include at least one of an imagesensor 120, a proximity sensor 120, and an infrared sensor 120. Thesecond sensor 120 may include at least one of an illuminance sensor 120and a color sensor 120. The sensor 120 may include various types ofsensors 120, such as a global positioning system (GPS) sensor 120, agyro sensor 120, an acceleration sensor 120, a LiDAR sensor 120, aninertial measurement unit (IMU) 120, a motion sensor 120, or the like.

At least one instruction regarding the electronic device 100 may bestored in the memory 130. In addition, an operating system (O/S) fordriving the electronic device 100 may be stored in the memory 130. Thememory 130 may store various software programs or applications foroperating the electronic device 100 according to various embodiments.The memory 130 may include a semiconductor memory such as a flashmemory, a magnetic storage medium such as a hard disk, or the like.

Specifically, the memory 130 may store various software modules foroperating the electronic device 100, and the processor 140 (e.g., atleast one processor) may control the operation of the electronic device100 by executing various software modules that are stored in the memory130. That is, the memory 130 may be accessed by the processor 140, andmay perform reading, recording, modifying, deleting, updating, or thelike, of data by the processor 140.

The term memory 130 may be used to refer to any volatile or non-volatilememory, a ROM, RAM proximate to or in the processor 140 or a memory card(for example, a micro secure digital (SD) card, a memory stick) mountedto the electronic device 100.

In various embodiments, the memory 130 may store various information ordata as described below. The memory 130 may store information about auser sensed through the first sensor 120, information on the directionand intensity of the light sensed through the second sensor 120,information on the position and size of the first area 21, informationon the position and size of the second area 22, information on theluminance of the second area 22, a plurality of content, or the like.The memory 130 may store various types of user data. Specific,non-limiting examples and embodiments of user data will be described indetail with reference to FIG. 9.

Various information required within a range for achieving the purpose ofthe disclosure may be stored in the memory 130, and information storedin the memory 130 may be received from an external device or updated bybeing received by a user.

The processor 140 may control overall operations of the electronicdevice 100. Specifically, the processor 140 may be connected to aconfiguration of the electronic device 100 including the mirror display110, the sensor 120, and the memory 130 as described above, and maycontrol overall operations of the electronic device 100 by executing atleast one instruction stored in the memory 130 as described above.

The processor 140 may be implemented in various ways. For example, theprocessor 140 may be implemented as at least one of an applicationspecific integrated circuit (ASIC), an embedded processor, amicroprocessor, a hardware control logic, a hardware finite statemachine (FSM), a digital signal processor (DSP), or the like. Further,processor 140 may include at least one of a central processing unit(CPU), a graphic processing unit (GPU), a main processing unit (MPU), orthe like.

According to an embodiment, the processor 140 may sense the user infront of the mirror display 110 through the first sensor 120. Theprocessor 140 may sense a user in front of the mirror display 110through at least one of the image sensor 120, the proximity sensor 120,and the infrared sensor 120 included in the first sensor 120.

As shown in FIGS. 3 through 8, when the user is located in front of themirror display 110, the first image 11 corresponding to the user may bereflected off of one area of the mirror display 110 as the lightincident from the external light source is reflected on the mirrordisplay 110. Hereinafter, as the light incident from the external lightsource is reflected on the mirror display 110, the user image reflectedoff of one area of the mirror display 110 is referred to as the “firstimage 11.”

When the user 10 in front of the mirror display 110 is sensed, theprocessor 140 may obtain information on a first area 21 on which areflection of the user is reflected off of the mirror display 110. Theinformation on the first area 21 may include information on the size ofthe first area 21 and information on the position of the first area 21.The location of the first area 21 may be based on a position of theuser, such as a position of the user's eyes. That is, the first area 21may be based on a perspective of a user.

The processor 140 may determine a second area 22 for displaying contentto be provided to the user 10 based on information about the first area21. The processor 140 may determine the position and size of the secondarea 22 for displaying the content on the basis of the information onthe size of the first are 21 and the information on the position of thefirst area 21. According to an embodiment, the second area 22 may bedetermined as an area among areas on the mirror display 110 that is notoverlapped with the first area 21.

The processor 140 may obtain information on light incident from anexternal light source 30 through the second sensor 120. The informationon the light may include information on the direction of the light andinformation on the intensity of the light.

When information about the light is obtained through the second sensor120, the processor 140 may determine the luminance of the second area 22based on the information about the light and display the content on thesecond area 22. According to an embodiment, the processor 140 maydetermine the luminance of the second area 22 so as to correspond to theintensity of light incident on the second area 22, and display thecontent with determined luminance on the second area 22.

Various embodiments based on the control of the processor 140 will befurther described with reference to FIGS. 2 to 9.

FIGS. 2, 3, and 4 show an operation of an electronic device according toan embodiment of the disclosure.

When the user 10 is not located in front of the mirror display 110, theprocessor 140 may control the mirror display 110 to display a standbyscreen 1 as shown in FIG. 2. For example, the standby screen 1 mayinclude information about the current time, as shown in FIG. 2, and mayinclude various information such as information on the current weather.

When the user 10 approaches the front of the mirror display 110 whilethe standby screen 1 is displayed, as shown in FIG. 3, the first image11 corresponding to a reflection of the user 10 may be reflected off ofone area of the mirror display 110 as the light incident from theexternal light source 30 is reflected on the mirror display 110. Whenthe first image 11 is displayed in one area of the mirror display 110,the processor 140 may stop displaying of the standby screen 1, as shownin FIG. 3.

The processor 140 may sense the user 10 in front of the mirror display110 through the first sensor 120. The processor 140 may obtain an imageof a scene in front of the mirror display 110 through the image sensor120 included in the first sensor 120, perform object recognition on theobtained image, and sense the user 10 in front of the mirror display110. The processor 140 may sense the user 10 in front of the mirrordisplay 110 through at least one of the proximity sensor 120 and theinfrared sensor 120 included in the first sensor 120.

When the user 10 in front of the mirror display 110 is sensed, theprocessor 140 may obtain information on the first area 21 of the mirrordisplay 110. The processor 140 may identify pixels of the mirror display110 corresponding to the first image 11 and identify the position andsize of the first area 21 including the identified pixels. As shown inFIG. 4, the first area 21 may be an area of the center of the mirrordisplay 110.

The processor 140 may determine a second area 22 for displaying contentto be provided to the user 10 based on information about the first area21. The processor 140 may determine the position and size of the secondarea 22 for displaying the content on the basis of the information onthe size of the first area 21 and the information on the position of thefirst area 21. As shown in FIG. 4, the content to be provided to theuser 10 may include health-related data related to health of the user10, and the second area 22 may be determined as a right top area of themirror display 110 among the areas on the mirror display 110 that arenot overlapped with the first area 21.

The processor 140 may obtain information on light incident from theexternal light source 30 through the second sensor 120. The informationon the light may include information on the direction of the light andinformation on the intensity of the light. When information about thelight is obtained through the second sensor 120, the processor 140 maydetermine the luminance of the second area 22 based on the informationabout the light and display the content on the second area 22 with thedetermined luminance.

The visibility of the user 10 with respect to the content displayed onthe second area 22 may vary depending on the direction and intensity ofthe light incident from the external light source 30. For example, whena strong intensity of light is incident on the mirror display 110 fromthe external light source 30 disposed in the right upper direction ofthe electronic device 100, the visibility of the user 10 with respect tothe content displayed on the second area 22 at the right upper end ofthe mirror display 110 may be lowered.

The processor 140 may adaptively determine the luminance of the secondarea 22 according to the direction and intensity of the light incidentfrom the external light source 30, thereby increasing the visibility ofthe user 10 with respect to the content displayed on the second area 22.The processor 140 may determine the luminance of the second area 22 soas to correspond to the intensity of light incident on the second area22. According to an embodiment, the processor 140 may determine thebrightness of the second area 22 to be proportional to the intensity oflight incident on the second area 22. According to another embodiment,data for a luminance level corresponding to each of a plurality oflevels according to the intensity of light may be prestored in thememory 130, and the processor 140 may determine the luminance of thesecond area 22 according to a luminance level corresponding to a levelof the intensity of light incident on the second area 22.

According to the embodiment described above with reference to FIGS. 2 to4, the electronic device 100 may determine an area for displayingcontent to be provided to the user 10 on the basis of the information onthe area in which the user image is displayed according to the mirrorfunction, and adjust the luminance of the area for displaying thecontent on the basis of the information on the light incident on themirror display 110, thereby improving visibility of the user 10 withrespect to the content displayed on the mirror display 110.

FIG. 5 is a diagram showing an embodiment for determining a second area.

Referring to FIGS. 2 to 4, the second area 22 is determined based on theinformation on the first area 21, but this is merely exemplary and it isnot necessary to use only the information about the first area 21 indetermining the second area 22 according to the disclosure. According toanother embodiment of the disclosure, the processor 140 may determinethe second area 22 using information on light incident from the externallight source 30 along with information about the first area 21.

As described above, when the user 10 approaches the front surface of themirror display 110, the first image 11 corresponding to the user 10 maybe reflected off of one area of the mirror display 110. When the user 10in front of the mirror display 110 is sensed, the processor 140 mayobtain information on the first area 21 of the mirror display 110. Theprocessor 140 may obtain information on light incident from the externallight source 30 through the second sensor 120.

When the information about the first area 21 and the information aboutthe light are obtained, the processor 140 may determine the position andsize of the second area 22 based on the information about the first area21 and the information about the light, and determine the luminance ofthe second area 22 based on the information about the light.

For example, as shown in FIG. 5, when light of strong intensity isincident on the mirror display 110 from the external light source 30disposed in the upper right direction of the electronic device 100, inparticular, the visibility of the user 10 with respect to the contentdisplayed on the upper right area of the display 110 may be reduced, andthe visibility of the user 10 with respect to the content displayed onthe lower left area of the minor display 110 may be relatively high.Accordingly, the processor 140 may select one area of the lower leftside of the mirror display 110 instead of the upper right side of theminor display 110 among the areas on the minor display 110 that do notoverlap the first area 21 as the second area 22.

However, the embodiment of FIG. 5 is merely an embodiment, and thesecond area 22 may be determined as an area of a left upper end or aright lower end of the mirror display 110, and at least a portion of thesecond area 22 may be determined to overlap at least a portion of thefirst area 21. If at least a portion of the second area 22 is determinedto overlap at least a portion of the first area 21, the processor 140may process the content such that the boundary area between the firstimage 11 reflected off of the first area 21 and the content displayed onthe second area 22 is blended.

The processor 140 may determine the second area 22 based on informationon the gaze of the user 10. The processor 140 may obtain information onthe gaze of the user 10 through the sensor 120, and determine one areaincluding the position on the mirror display 110 corresponding to thegaze of the user 10 as the second area based on the information on thegaze of the user 10.

According to the embodiment described above with reference to FIG. 5,the electronic device 100 may improve the visibility of the user 10 withrespect to the content displayed on the minor display 110 by determiningthe area for displaying the content to be provided to the user 10 usingthe information on the area in which the user image is displayedaccording to the minor function and information about light incident onthe minor display 110.

FIG. 6 is a diagram showing an embodiment for determining a first area.

Based on assumption that the motion of the user 10 in front of themirror display 110 is not large, a method of obtaining information aboutthe first area 21 in which the first image 11 is reflected off of themirror display 110 is described, but if a motion of the user 10 islarge, an area on which content is displayed by the user image reflectedoff of the mirror display 110 may be largely restricted. Accordingly,the processor 140 may determine a first area 21 for displaying contentusing information about the pose of the user 10.

In detail, the first sensor 120 may include the image sensor 120, andthe processor 140 may obtain a second image corresponding to the user 10through the image sensor 120. The image sensor 120 may refer to thesensor 120 capable of converting light entering through the lens of thecamera into an electrical image signal, and may be included as onecomponent of the camera. In the disclosure, obtaining the second imagethrough the image sensor 120 may be replaced by obtaining a second imagethrough the camera.

The user image obtained through the image sensor 120 is referred to as“second image” by distinguishing from the term first image 11 whichrefers to a user image reflected off of an area of the mirror display110 as the light incident from the external light source 30 is reflectedon the mirror display 110.

Once the second image is obtained, the processor 140 may input thesecond image into a trained first neural network model to obtaininformation about the pose of the user 10. The “first neural networkmodel” refers to a model trained to obtain information about the pose ofthe user 10 included in the user image based on the inputted user image.Specifically, the processor 140 may detect the user 10 included in thesecond image through the first neural network model and obtain imagedata for an area including the user 10. The processor 140 may obtainpose information for the user 10 based on information on a feature pointor skeleton of the user 10 included in the image data.

If information about the pose of the user 10 is obtained as describedabove, the processor 140 may determine the position and size of thefirst area 21 based on the information on the pose of the user 10. Theprocessor 140 may estimate an area in which the first image 11 isdisplayed according to the pose of the user 10 during a predeterminedperiod on the basis of the information about the pose of the user 10,and determine the estimated area as the first area 21. For example, asshown in FIG. 6, the processor 140 may estimate that the first image 11may reach the upper portion of the mirror display 110 during apredetermined period based on information about the pose of the user 10,thereby determining the area 21 as shown in FIG. 6 as the second area22. Here, the predetermined period may be changed by a developer orsetting of the user 10.

As described above, if the first area 21 is determined based on theinformation on the pose of the user 10, the processor 140 may determinethe second area 22 for displaying the content based on the informationabout the first area 21, determine the luminance of the second area 22based on the information about the light, and display the content on thesecond area 22 with the determined luminance. For example, as shown inFIG. 6, the second area 22 may be determined as the right top area ofthe mirror display 110 among the areas on the mirror display 110 thatare not overlapped with the first area 21.

The processor 140 may identify the content to be provided to the user 10based on information about the pose of the user 10. The processor 140may estimate an exercise motion of the user 10 based on informationabout the pose of the user 10 and identify content corresponding to theestimated motion. For example, as shown in FIG. 6, if the user 10 isassumed to perform a “lunge” movement on the basis of the informationabout the pose of the user 10, the processor 140 may identifyrecommended content for guiding the “lunge” movement and control themirror display 110 to display the identified recommendation content onthe second area 22.

Referring to FIG. 6, the electronic device 100 may determine an area inwhich the user image is displayed according to a mirror function duringa predetermined interval on the basis of the information about the poseof the user 10, and in particular, when the movement of the user 10 islarge, it is possible to prevent the area displayed on the mirrordisplay 110 from being restricted by the user image reflected off of themirror display 110.

FIG. 7 is a diagram showing an embodiment for determining brightnessdifferently for each object included in the content.

In the above, information on the light incident from the external lightsource 30 may be obtained through the second sensor 120, and the contentis displayed by determining the luminance of the second area 22 tocorrespond to the intensity of light incident on the second area 22,more specifically, the luminance of the second area 22 may be determineddifferently depending on the object included in the content.Hereinafter, in the description of FIG. 7, similarly to the descriptionof FIG. 6, the second area 22 is determined as an upper right area ofthe mirror display 110 among the areas on the mirror display 110 whichis not overlapped with the first area 21.

Specifically, as shown in FIG. 7, the content displayed on the secondarea 22 may include a plurality of objects. As an example, the pluralityof objects may be divided into a “person” and a “background”. Theprocessor 140 may input data on the content into the trained secondneural network model to identify the types of each of the plurality ofobjects included in the content. Here, the “second neural network model”is a neural network trained to output information on the probabilitythat a plurality of objects included in the content correspond to eachof a plurality of predefined classes (or categories, domains) based onthe data on the content.

The processor 140 may determine the luminance of the second area 22 foreach area corresponding to each of the plurality of objects included inthe content based on the information about the light and the type ofeach of the plurality of objects. For example, as shown in FIG. 7, whenthe external light source 30 is disposed on the right upper end of theelectronic device 100, the visibility of the user 10 with respect to thecontent displayed on the second area 22 on the right upper end of themirror display 110 may deteriorate. As shown in FIG. 7, the processor140 may determine the luminance of a “person” among the plurality ofobjects included in the content to be high, and determine the luminanceof the “background” to be low.

The processor 140 may determine the luminance of the second area 22based on the information on the light, the information on the type ofcontent, and the brightness of the content. The memory 130 may storeinformation about the type of the content (or genre) and luminancecorresponding to the brightness, and the processor 140 may determine theluminance of the second area 22 by using the information on theluminance corresponding to the type and brightness of the contenttogether with the information on the light. For example, if the contentis “battle movie” and the average brightness of the image frame includedin the content is less than a predetermined threshold value, theprocessor 140 may determine the luminance of the second area 22relatively lower than when the luminance of the second area 22 isdetermined based on only information about the light.

According to the embodiment described above with reference to FIG. 7,the electronic device 100 may substantially improve the user's 10visibility of the content by differently determining the luminance of anarea in which the content is displayed according to an object includedin the content.

FIG. 8 shown an operation of displaying a user interface for controllingcontent, according to an embodiment.

In the above description, the first image 11 corresponding to the user10 in front of the mirror display 110 is reflected off of the first area21 and the content to be provided to the user 10 is displayed on thesecond area 22, but a user interface (UI) for controlling the contentmay be displayed on the mirror display 110. Hereinafter, as shown inFIG. 8, the first area 21 is determined as the center area of the mirrordisplay 110 and the second area 22 is determined as the right upper areaof the mirror display 110 among the areas on the mirror display 110,which is not overlapped with the first area 21.

The processor 140 may control the mirror display 110 to determine theposition of the user 10 interface for controlling the content based onthe information about the first area 21 and the information about thesecond area 22, and to display the user 10 interface at the determinedposition. For example, the processor 140 may control the mirror display110 to display the user 10 interface on an area overlapping a portion ofthe first area 21 where the first image 11 corresponding to the user 10is provided to facilitate the touch input of the user 10.

The user interface may include a plurality of UI items 23-1, 23-2, 23-3,23-4, 23-5 for controlling the content displayed on the second area 22.

As shown in FIG. 8, the user interface may include a UI item 23-1corresponding to a “rewind” function, a UI item 23-2 corresponding to afunction of “moving to a previous content”, a UI item 23-3 correspondingto a function of “moving to the next content”, a UI item 23-4corresponding to a “fast forward” function, and a UI item 23-5corresponding to “play/pause” function. The UI item 23-2 correspondingto a function of “moving to the previous content” and the UI item 23-5corresponding to the “play/pause” function may overlap a portion of thefirst area 21 on which the first image 11 is displayed. In this example,the processor 140 may process an image corresponding to the UI items23-2 and 23-5 such that a boundary area between the UI items 23-2 and23-5 is blended, thereby harmonizing the UI item with the first image11.

The processor 140 may process an image corresponding to the UI items23-2 and 23-5 such that the boundary area between the first image 11 andthe UI items 23-2 and 23-5 may be blended, but may process the contentto be blended with the boundary area between the first image 11 and thecontent. The processor 140 may process the content so that a boundaryarea between the first area 21 and the second area 22 is blended, andcontrol the mirror display 110 to display the processed content on thesecond area 22.

According to the embodiment described above with reference to FIG. 8,the electronic device 100 may display a user image reflected off of themirror display 110 according to a mirror function, and a user interfacefor controlling the content together with the content to be provided tothe user 10, thereby improving the convenience of the user 10.

FIG. 9 is a block diagram showing details of an electronic deviceaccording to an embodiment.

As shown in FIG. 9, the electronic device 100 according to an embodimentmay further include a communication interface 150, a microphone 160, anda speaker 170 as well as the mirror display 110, the sensor 120, thememory 130, and the processor 140. The mirror display 110, the sensor120, the memory 130, and the processor 140 have been described abovewith reference to FIGS. 1 to 8, and a duplicate description will beomitted. However, prior to describing the communication interface 150,the microphone 160, and the speaker 170, the “user data” stored in thememory 130 and the embodiments related thereto will be described.

As shown in FIG. 9, the memory 130 may store user data according to thedisclosure. Here, “user data” is used to collectively refer to variousdata related to the user 10 of the electronic device 100. Specifically,the user data may include identification information, personalinformation, and health-related information of the user 10.

The “identification information” may include a face image, fingerprintdata, iris data, and account information for identifying the user 10.The “user personal information” may include information about thepersonal information of the user 10 and information about a schedule ofthe user 10, or the like. The “health-related information” may includeinformation on the heart rate of the user 10, information on the bodytemperature of the user 10, information on the weight of the user 10,information on the body component of the user 10, information on thesleep pattern of the user 10, information on the stress of the user 10,information on the hormone of the user, information on the emotion ofthe user 10, information on the exercise history of the user 10,information on the exercise history of the user 10, and the like.

The user data may be obtained through various types of sensors 120included in the electronic device 100, may be obtained through varioustypes of sensors 120 included in the external device, and may bereceived from an external device through the communication interface 150of the electronic device 100. Here, the external device may be a userterminal, such as a smart phone, a smart watch, or the like, or may be acloud server or an edge computing device configured to store/manage userdata.

According to an embodiment, the processor 140 may identify whether theuser 10 in front of the mirror display 110 is the pre-registered user 10based on the user data. The processor 140 may identify whether the user10 of the front surface of the mirror display 110 is the pre-registereduser 10 depending on whether the identification information stored inthe memory 130 is matched with the identification information (e.g.,face image, fingerprint data, or iris data) obtained through the sensor120. In addition, the processor 140 may identify whether the user 10 isthe pre-registered user 10 according to whether the account informationstored in the memory 130 matches the account information input by theuser 10.

When it is identified that the user 10 of the front surface of themirror display 110 is not the pre-registered user 10, the processor 140may control the mirror display 110 to display a basic screen includinginformation on the current time, the current weather, or the like.

When the user 10 of the front surface of the mirror display 110 isidentified as the pre-registered user 10, the processor 140 may outputthe greeting message based on the personal information of the user 10.For example, the processor 140 may control the mirror display 110 todisplay a greeting message, such as “Hello, Mr. Kim”, and may output avoice signal corresponding to the “Hello, Mr. Kim” through the speaker170.

If the user 10 of the front surface of the mirror display 110 is thepre-registered user 10, the processor 140 may control the mirror display110 to display health related data corresponding to the pre-registereduser 10. The processor 140 may control the mirror display 110 to displayinformation on the heart rate of the user 10, information on the bodytemperature of the user 10, information on the weight of the user 10,information on the body component of the user 10, information on thesleep pattern of the user 10, information on the stress of the user 10,information on the hormone of the user, information on the emotion ofthe user 10, information on the exercise history of the user 10,information on the exercise history of the user 10, and the like, who isidentified as the pre-registered user 10.

If the user 10 on the front of the mirror display 110 is thepre-registered user 10, the processor 140 may identify the content to beprovided to the user 10 based on health-related data corresponding tothe pre-registered user 10 and control the mirror display 110 to displaythe identified content on the second area 22. The processor 140 mayidentify content such as content that has been recently played by theuser 10 that is identified as the pre-registered user 10 among thecontent stored in the memory 130, content set to be preferred by theuser 10, recommendation content corresponding to at least some of thehealth-related information of the user 10, and to display at least onecontent of the identified content on the second area 22.

The process of identifying the recommended content to be provided to theuser 10 based on the user data may be performed through the trainedthird neural network model. The “third neural network model” is trainedto identify content corresponding to the personal information and thehealth-related data of the user 10, and the electronic device 100 mayidentify recommended content to be provided to the user 10 using thethird neural network model.

While the content provided to the user 10 is stored in the memory 130 ofthe electronic device 100, the content provided to the user 10 may bestored in the external device, and the electronic device 100 may receivethe content from the external device and may provide the receivedcontent to the user 10.

According to an embodiment as described above, when the user 10 of thefront surface of the mirror display 110 is the pre-registered user 10,the electronic device 100 may provide personalized health-related dataand content to the user 10 of the front surface of the mirror display110 based on the user data, thereby contributing to the health conditionmanagement/improvement of the user 10.

The communication interface 150 may include a circuit and may performcommunication with an external device. The processor 140 may receivevarious data or information from an external device connected throughthe communication interface 150, and may transmit various data orinformation to an external device.

The communication interface 150 may include at least one of a Wi-Fimodule, a Bluetooth module, a wireless communication module, and a nearfield communication (NFC) module. To be specific, the Wi-Fi module maycommunicate by a Wi-Fi method and the Bluetooth module may communicateby a Bluetooth method. When using the Wi-Fi module or the Bluetoothmodule, various connection information such as service set identifier(SSID) may be transmitted and received for communication connection andthen various information may be transmitted and received.

The wireless communication module may communicate according to variouscommunication specifications such as IEEE, Zigbee, 3rd generation (3G),3rd generation partnership project (3GPP), long term evolution (LTE),5th generation (5G), or the like. The NFC module may communicate by theNFC method using a 13.56 MHz band among various RF-ID frequency bandssuch as 135 kHz, 13.56 MHz, 433 MHz, 860-960 MHz, 2.45 GHz, or the like.

According to various embodiments, the processor 140 may receive userdata as described above from an external device through thecommunication interface 150 and may store the received user data in thememory 130.

The microphone 160 may receive a voice signal and convert the receivedvoice signal to an electrical signal. According to an embodiment, theprocessor 140 may receive a voice signal corresponding to a voice of theuser 10, input the received voice signal to a trained voice recognitionmodel to obtain text information corresponding to the voice signal, andidentify a control command corresponding to the obtained textinformation. For example, the user voice may correspond to a controlcommand to change the position of the second area 22 or the size of thesecond area 22, a control command to change the type of contentdisplayed on the second area 22, or a control command to control contentdisplayed on the second area 22.

The speaker 170 may output audio data under the control of the processor140. According to an embodiment, the speaker 170 may output a voicesignal corresponding to the content displayed on the second area 22, afeedback voice signal corresponding to the voice of the user 10 receivedthrough the microphone 160, or a greeting message for the pre-registereduser 10.

Although various configurations of the electronic device 100 have beendescribed above, the above-described configurations are merelyexemplary, and in the practice of the disclosure, a new configurationmay be added or some configuration may be omitted in addition to theconfiguration as shown in FIG. 9.

FIG. 10 is a flowchart of a method of controlling an electronic deviceaccording to an embodiment.

As shown in FIG. 10, the electronic device 100 may sense the user infront of the mirror display in operation S1010. When the user in frontof the mirror display is not detected in operation S1010-N, theelectronic device 100 may display a standby screen until a user of thefront surface of the mirror display is sensed.

If the user of the front surface of the mirror display is sensed inoperation S1010-Y, the electronic device 100 may obtain information onthe first area in which the first image corresponding to a reflection ofa user is reflected off of the mirror display in operation S1020. Theinformation on the first area may include information on the size of thefirst area and information on the position of the first area.

The electronic device 100 may determine a second area for displaying thecontent to be provided to the user based on the information about thefirst area in operation S1030. The electronic device 100 may determine aposition and a size of a second area for displaying content on the basisof information on the size of the first area and information on aposition of the first area. According to an embodiment, the second areamay be determined as one of the areas on the mirror display that is notoverlapped with the first area.

The electronic device 100 may obtain information on light incident fromthe external light source through the second sensor. The information onthe light may include information on the direction of the light andinformation on the intensity of the light.

When information about the light is obtained through the second sensor,the electronic device 100 may determine the luminance of the second areabased on the information about the light in operation S1040. Theelectronic device 100 may display the content with the determinedluminance on the second area in operation S1050. According to anembodiment, the processor may determine the luminance of the second areato correspond to the intensity of light incident on the second area, anddisplay the content with the determined luminance on the second area.

The controlling method of the electronic device 100 according to theaforementioned embodiment may be implemented as a program and providedto the electronic device 100. In particular, a program including thecontrolling method of the electronic device 100 may be stored in anon-transitory computer readable medium and provided.

Specifically, in a computer-readable recording medium including aprogram for executing a controlling method of the electronic device 100,the controlling method of the electronic device 100 may include thesteps of obtaining information on a first area on which a first imagecorresponding to a user is reflected off of a mirror display when a userof the front surface of the mirror display is sensed; determining asecond area for displaying content to be provided to the user on thebasis of the information on the first area; determining luminance of thesecond area on the basis of the information on the light when theinformation on the light incident from the external light source isobtained; and displaying the content with the determined luminance onthe second area.

As described above, the electronic device 100 may determine a secondarea using information on light incident from an external light sourcealong with information about the first area, determine a first area fordisplaying the content using information about the pose of the user, andthe luminance of the second area may be determined differently accordingto an object included in the content.

The non-transitory computer readable medium refers to a medium thatstores data semi-permanently rather than storing data for a very shorttime, such as a register, a cache, a memory, etc., and is readable by anapparatus (i.e., executable by at least one processor). In detail, theaforementioned various applications or programs may be stored in thenon-transitory computer readable medium, for example, a compact disc(CD), a digital versatile disc (DVD), a hard disc, a Blu-ray disc, auniversal serial bus (USB), a memory card, a read only memory (ROM), andthe like, and may be provided.

A controlling method of the electronic device 100 and acomputer-readable recordable medium including a program for executingthe controlling method of the electronic device 100 are described, butthis is to omit redundant description, and various embodiments of theelectronic device 100 may be applied to the controlling method of theelectronic device 100 and a computer-readable recordable mediumincluding a program for executing the controlling method of theelectronic device 100.

A function related to the foregoing may be performed by a memory and aprocessor.

The processor may include one or a plurality of processors. At thistime, one or a plurality of processors may be a general purposeprocessor, such as a central processing unit (CPU), an applicationprocessor (AP), or the like, a graphics-only processing unit such as agraphics processing unit (GPU), a visual processing unit (VPU), and/oran AI-dedicated processor such as a neural processing unit (NPU).

The one or a plurality of processors control the processing of the inputdata in accordance with a predefined operating rule or artificialintelligence (AI) model stored in the non-volatile memory and thevolatile memory. The predefined operating rule or artificialintelligence model is provided through training or learning.

Being provided through learning may refer, for example, to, by applyinga learning algorithm to a plurality of learning data, a predefinedoperating rule or AI model of a desired characteristic being made. Thelearning may be performed in a device itself in which AI according to anembodiment is performed, and/o may be implemented through a separateserver/system.

The AI model may include a plurality of neural network layers. Eachlayer has a plurality of weight values, and performs a layer operationthrough calculation of a previous layer and an operation of a pluralityof weights. Examples of neural networks include, but are not limited to,convolutional neural network (CNN), deep neural network (DNN), recurrentneural network (RNN), restricted Boltzmann Machine (RBM), deep beliefnetwork (DBN), bidirectional recurrent deep neural network (BRDNN),generative adversarial networks (GAN), and deep Q-networks.

The learning algorithm may include a method for training a predeterminedtarget device (for example, a robot) using a plurality of learning datato cause, allow, or control the target device to make a determination orprediction. Examples of learning algorithms include, but are not limitedto, supervised learning, unsupervised learning, semi-supervisedlearning, or reinforcement learning.

The machine-readable storage medium may be provided in the form of anon-transitory storage medium. The, “non-transitory” storage medium maynot include a signal (e.g., electromagnetic wave) and is tangible, butdoes not distinguish whether data is permanently or temporarily storedin a storage medium. For example, the “non-transitory storage medium”may include a buffer in which data is temporarily stored.

According to various embodiments, a method disclosed herein may beprovided in a computer program product. A computer program product maybe traded between a seller and a purchaser as a commodity. A computerprogram product may be distributed in the form of a machine readablestorage medium (e.g., compact disc ROM (CD-ROM)) or distributed onlinethrough an application store (e.g., PlayStore™) or distributed (e.g.,download or upload) online between two user devices (e.g., smartphones)directly. In the case of on-line distribution, at least a portion of thecomputer program product (e.g., a downloadable app) may be storedtemporarily or at least temporarily in a storage medium such as amanufacturer's server, a server in an application store, or a memory ina relay server.

In addition, each of the components (e.g., modules or programs)according to various embodiments may include a single entity or aplurality of entities, and some sub-components of the sub-componentsdescribed above may be omitted, or other sub-components may be furtherincluded in the various embodiments. Alternatively or additionally, somecomponents (e.g., modules or programs) may be integrated into one entityto perform the same or similar functions performed by the respectivecomponents prior to the integration.

The operations performed by the module, the program, or other component,in accordance with various embodiments may be performed in a sequential,parallel, iterative, or heuristic manner, or at least some operationsmay be executed in a different order or omitted, or other operations maybe added.

The term “unit” or “module” used in the disclosure includes unitsconsisting of hardware, software, or firmware, and is usedinterchangeably with terms such as, for example, logic, logic blocks,parts, or circuits. A “unit” or “module” may be an integrallyconstructed component or a minimum unit or part thereof that performsone or more functions. For example, the module may be configured as anapplication-specific integrated circuit (ASIC).

Embodiments may be implemented as software that includes instructionsstored in machine-readable storage media readable by a machine (e.g., acomputer). A device may call instructions from a storage medium and thatis operable in accordance with the called instructions, including anelectronic device (e.g., the electronic device 100).

When the instruction is executed by a processor, the processor mayperform the function corresponding to the instruction, either directlyor under the control of the processor, using other components. Theinstructions may include a code generated or executed by the compiler orinterpreter.

While embodiments of the disclosure have been shown and described, itwill be understood by those skilled in the art that various changes inform and details may be made therein without departing from the spiritand scope of the disclosure as defined at least by the appended claimsand their equivalents.

What is claimed is:
 1. An electronic device comprising: a first sensorconfigured to sense a user; a second sensor configured to sense lightincident from an external light source; a mirror display; and aprocessor configured to: based on the first sensor sensing the user infront of the mirror display, obtain information about a first area inwhich a first image corresponding to a reflection of the user is locatedon the mirror display, determine a second area for displaying content tobe provided to the user based on the information about the first area,based on information about light being obtained through the secondsensor, determine luminance of the second area based on the informationabout the light, and control the mirror display to display the contentwith the determined luminance on the second area.
 2. The electronicdevice of claim 1, wherein the information about the first areacomprises information about a size of the first area and informationabout a position of the first area, and wherein the processor is furtherconfigured to determine a position and a size of the second area basedon the information about the size of the first area and the informationabout the position of the first area.
 3. The electronic device of claim1, wherein the information about the light comprises information about adirection of the light and information about an intensity of the light,and wherein the processor is further configured to: determine a positionand a size of the second area based on the information about the firstarea and the information about the direction of the light, and determineluminance of the second area based on the information about theintensity of the light.
 4. The electronic device of claim 1, wherein thefirst sensor comprises an image sensor, and wherein the processor isfurther configured to: obtain a second image corresponding to the userthrough the image sensor, obtain information about a pose of the user byinputting the second image into a trained neural network model, anddetermine a position and a size of the first area based on theinformation about the pose of the user.
 5. The electronic device ofclaim 1, wherein the processor is further configured to: identify typesof a plurality of objects within the content, respectively, by inputtingdata about the content to a trained neural network model, and determineluminance of areas corresponding to each of the plurality of objects inthe second area based on the information about the light and the typesof each of the plurality of objects.
 6. The electronic device of claim1, wherein the processor is further configured to determine luminance ofthe second area based on the information about the light, theinformation about types of the content, and information about brightnessof the content.
 7. The electronic device of claim 1, wherein theprocessor is further configured to: determine a position of a userinterface to control the content based on the information about thefirst area and the information about the second area, and control themirror display to display the user interface at the determined position.8. The electronic device of claim 1, further comprising: a memory,wherein the processor is further configured to: identify whether theuser is a pre-registered user based on user data stored in the memory,based on the user being the pre-registered user, identify the contentbased on the user data corresponding to the pre-registered user, andcontrol the mirror display to display the identified content on thesecond area.
 9. The electronic device of claim 8, further comprising: acommunication interface, wherein the user data is health-related datareceived from an external device through the communication interface,and wherein the content comprises recommended content corresponding tothe health-related data.
 10. The electronic device of claim 1, whereinthe processor is further configured to: process the content so that aboundary area between the first area and the second area is blended, andcontrol the mirror display to display the processed content on thesecond area.
 11. A method of controlling an electronic device comprisinga mirror display, the method comprising: based on a user on in front ofthe mirror display being sensed, obtaining information about a firstarea in which a first image corresponding to a reflection of the user islocated on the mirror display; determining a second area for displayingcontent to be provided to the user based on the information about thefirst area; based on information about light being obtained through asecond sensor, determining luminance of the second area based on theinformation about the light; and displaying the content with thedetermined luminance on the second area.
 12. The method of claim 11,wherein the information about the first area comprises information abouta size of the first area and information about a position of the firstarea, and wherein the determining the second area comprises determininga position and a size of the second area based on the information aboutthe size of the first area and the information about the position of thefirst area.
 13. The method of claim 11, wherein the information aboutthe light comprises information about a direction of the light andinformation about an intensity of the light, wherein the determining thesecond area comprises determining a position and a size of the secondarea based on the information about the first area and the informationabout the direction of the light, and wherein the determining theluminance of the second area comprises determining luminance of thesecond area based on the information about the intensity of the light.14. The method of claim 11, further comprising: obtaining a second imagecorresponding to the user; obtaining information about a pose of theuser by inputting the second image to a trained neural network model;and determining a position and a size of the first area based on theinformation about the pose of the user.
 15. The method of claim 11,wherein the content comprises content including a plurality of objects,and wherein the determining the second area further comprises:identifying types of the plurality of objects, respectively, byinputting data about the content to a trained neural network model; anddetermining luminance of areas corresponding to each of the plurality ofobjects in the second area based on the information about the light andthe types of each of the plurality of objects.
 16. An electronic devicecomprising: a first sensor configured to sense a user; a second sensorconfigured to sense light incident from an external light source; amirror display configured to provide a visible reflection of the userwhile also displaying content; and a processor configured to: based onthe first sensor sensing the user in front of the mirror display, obtaininformation about a location of a reflection of the user from aperspective of the user; determine a displaying area for displayingcontent to be provided to the user based on information about thelocation of the reflection of the user; and control the mirror displayto display the content on the displaying area.